Cloud Computing for Fundamental Spatial Operations on Polygonal GIS Data

Efficient end-to-end parallel/distributed processing of polygon-based spatial data (also known as vector-based data) has been a long-standing research question in GIS community. The irregular and data intensive nature of the underlying computation has impeded the exploratory research in this space. We have created an open architecture based system named Crayons for Azure cloud platform using state-of-the-art techniques. The design and development of Crayons system is an engineering feat both due to (i) the emerging nature of the Azure cloud platform which lacks traditional support for parallel processing and (ii) the tedious exploration of design space for right techniques for parallelizing various workflow components including file I/O, partitioning, task creation, and load balancing. Crayons is an open-source system available for both download and online access, to foster academic activities. We believe Crayons to be the first distributed GIS system over cloud capable of end-to-end spatial overlay analysis. We demonstrate how Azure platform’s storage, communication, and computation mechanisms can support high performance application (HPC) development. Crayons scales well for sufficiently large data sets, achieving end-toend relative speedup of over 40-fold employing 100 Azure processors. For smaller, more irregular workload, it still yields over 10-fold speedup.